Passenger counting device, system, method and program, and vehicle movement amount calculation device, method and program

ABSTRACT

The erroneous detection determination system acquires a movement amount of a vehicle. The erroneous detection determination system calculates, based on the movement amount of the vehicle, distances in a depth direction of objects detected as faces of passengers of the vehicle from an image of the vehicle. The erroneous detection determination system determines whether the detected objects include an object that has been erroneously detected, based on the distances in the depth direction of the objects.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.15/572,826, U.S. Pat. No. 10,318,829, filed Nov. 9, 2017, which is aNational Stage of International Application No. PCT/JP2016/001548 filedMar. 17, 2016, the disclosures of which are incorporated herein in theirentirety by reference.

TECHNICAL FIELD

The present invention relates to passenger counting device, system,method, and program to count the number of passengers of a vehicle, andvehicle movement amount calculation device, method, and program.

BACKGROUND ART

In recent years, a high occupancy vehicle (HOV) system, which discountstolls depending on the number of passengers of a vehicle or permitspassage of a road only to a vehicle with the passenger number exceedinga predetermined number, has been used. In the HOV system, a technique isused in which the passenger number is counted by photographing a vehicleusing an installed camera and performing face detection on thephotographed image.

PTLs 1 to 3 are disclosed as a system for counting the number ofpassengers of a vehicle by face detection. PTL 1 discloses a techniqueof counting the number of passengers of a vehicle by detecting a profileof a person. PTL 2 discloses a technique of measuring the passengernumber by detecting persons and estimating at which positions in avehicle the persons are on board. PTL 3 discloses a technique ofcounting the passenger number using a movement amount of a vehicle and aperson detection result.

CITATION LIST Patent Literature

PTL 1: International Publication No. 2014/061195

PTL 2: International Publication No. 2014/064898

PTL 3: International Publication No. 2015/052896

SUMMARY OF INVENTION Technical Problem

In the systems disclosed in PTLs 1 to 3, there is a possibility oferroneous detection when face detection is performed. For example, theabove-described systems sometimes erroneously detect a plurality ofpersons at a close distance on an image as one person.

Accordingly, an object of the present invention is to provide passengercounting device, system, method and program, and vehicle movement amountcalculation device, method, and program capable of improving accuracy incounting of the number of passengers of a vehicle.

Solution to Problem

A passenger counting device according to the present invention ischaracterized by including: a movement amount calculation means forcalculating a movement amount of a vehicle based on an image of thevehicle; a depth distance calculation means for calculating a distancein a depth direction of a face of a passenger of the vehicle based onthe movement amount of the vehicle; and a passenger number determinationmeans for detecting the face of the passenger of the vehicle from theimage and determining the number of passengers of the vehicle based ondistances in the depth direction of a plurality of detected faces of thepassengers.

A passenger counting system according to the present invention ischaracterized by including: a photographing means for capturing avehicle and acquiring an image; a movement amount calculation means forcalculating a movement amount of the vehicle based on the image of thevehicle; a depth distance calculation means for calculating a distancein a depth direction of a face of a passenger of the vehicle based onthe movement amount of the vehicle; and a passenger number determinationmeans for detecting the face of the passenger of the vehicle from theimage and determining the number of passengers of the vehicle based ondistances in the depth direction of a plurality of detected faces of thepassengers.

A passenger counting method according to the present invention, themethod is characterized by calculating a movement amount of a vehiclebased on an image of the vehicle; calculating a distance in a depthdirection of a face of a passenger of the vehicle based on the movementamount of the vehicle, and detecting the face of the passenger of thevehicle from the image and determining the number of passengers of thevehicle based on distances in the depth direction of a plurality ofdetected faces of the passengers.

A passenger counting program according to the present invention ischaracterized by causing a computer to execute: a movement amountcalculation process of calculating a movement amount of a vehicle basedon an image of the vehicle; a depth distance calculation process ofcalculating a distance in a depth direction of a face of a passenger ofthe vehicle based on the movement amount of the vehicle; and a passengernumber determination process of detecting the face of the passenger ofthe vehicle from the image and determining the number of passengers ofthe vehicle based on distances in the depth direction of a plurality ofdetected faces of the passengers.

A vehicle movement amount calculation device according to the presentinvention is characterized by calculating a movement amount of a vehiclebased on an image of the vehicle and estimating an error of the movementamount of the vehicle using a steepest descent method.

A vehicle movement amount calculation method according to the presentinvention is characterized by calculating a movement amount of a vehiclebased on an image of the vehicle and estimating an error of the movementamount of the vehicle using a steepest descent method.

A vehicle movement amount calculation program according to the presentinvention is characterized by causing a computer to execute: a processof calculating a movement amount of a vehicle based on an image of thevehicle and estimating an error of the movement amount of the vehicleusing a steepest descent method.

Advantageous Effects of Invention

According to the present invention, it is possible to improve theaccuracy in counting of the number of passengers of the vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 It depicts a block diagram illustrating a configuration of afirst exemplary embodiment of a passenger counting system according tothe present invention.

FIG. 2 It depicts a flowchart illustrating an operation of the firstexemplary embodiment of the passenger counting system according to thepresent invention.

FIG. 3 It depicts an explanatory diagram illustrating calculation of anestimated movement amount performed by a movement amount calculationunit.

FIG. 4 It depicts an explanatory diagram illustrating a pinhole cameramodel.

FIG. 5 It depicts an explanatory diagram illustrating a method ofcalculating a depth distance from the camera to a target face.

FIG. 6 It depicts a schematic block diagram illustrating a configurationexample of a computer according to the present exemplary embodiment.

FIG. 7 It depicts a block diagram illustrating a configuration of asecond exemplary embodiment of the passenger counting system accordingto the present invention.

FIG. 8 It depicts a flowchart illustrating an operation of the secondexemplary embodiment of the passenger counting system according to thepresent invention.

FIG. 9 It depicts an enlarged view of a face of a passenger of avehicle.

FIG. 10 It depicts a block diagram illustrating a configuration of athird exemplary embodiment of the passenger counting system according tothe present invention.

FIG. 11 It depicts a flowchart illustrating an operation of the thirdexemplary embodiment of the passenger counting system according to thepresent invention.

FIG. 12 It depicts a block diagram illustrating a configuration of amain part of the passenger counting system according to the presentinvention.

DESCRIPTION OF EMBODIMENTS First Exemplary Embodiment

A passenger counting system according to the present exemplaryembodiment will be described with reference to the drawings. FIG. 1 is ablock diagram illustrating a configuration of the passenger countingsystem according to the present exemplary embodiment. The passengercounting system includes a photographing unit 10 and a passengercounting device 100. In addition, the passenger counting device 100includes a movement amount calculation unit 11, a depth distancecalculation unit 12, and a passenger number determination unit 13.

The photographing unit 10 photographs a vehicle and acquires an image.In the present exemplary embodiment, the photographing unit 10 is ageneral camera, and photographs a subject to generate a digital image.In addition, the photographing unit 10 is installed on a road side, andperforms photographing from a lateral direction of the vehicle (adirection substantially perpendicular to a traveling direction) in thepresent exemplary embodiment. The photographing unit 10 may employ acharge-coupled device (CCD) camera, a complementarymetal-oxide-semiconductor (CMOS) camera, an infrared camera, or thelike.

The movement amount calculation unit 11 calculates the movement amountof the vehicle based on the image of the vehicle acquired by thephotographing unit 10. Specifically, the movement amount calculationunit 11 estimates the movement amount of the vehicle based on themovement amount of a specific part of the vehicle such as a handleportion of a door.

The depth distance calculation unit 12 calculates a distance (depthdistance) in a depth direction of a face of a passenger of the vehiclein real space based on the movement amount of the vehicle calculated bythe movement amount calculation unit 11. Specifically, the depthdistance calculation unit 12 calculates the distance in the depthdirection from the photographing unit 10 to the face of the passengerbased on the calculated movement amount of the vehicle and a directiontoward the face of the passenger of the vehicle from a position of thephotographing unit 10 that has photographed the vehicle.

The passenger number determination unit 13 detects the face of thepassenger of the vehicle from the image of the vehicle acquired by thephotographing unit 10, determines presence or absence of erroneousdetection based on a distance in the depth direction between a pluralityof detected faces of passengers, and determines the number of passengersof the vehicle. For example, when the distance in the depth directionbetween the plurality of detected faces of passengers is equal to orlonger than a first threshold value or when a distance in the travelingdirection between the plurality of faces of passengers is equal to orlonger than a second threshold value, the passenger number determinationunit 13 determines that the plurality of faces of passengers are facesof different persons. The passenger number determination unit 13 may usea distance between partial images from which the face is detected as thedistance between the faces.

Next, an operation of the passenger counting system according to thepresent exemplary embodiment will be described. FIG. 2 is a flowchartillustrating the operation of the passenger counting system according tothe present exemplary embodiment.

The photographing unit 10 photographs a moving vehicle at a plurality oftimings and acquires images (step S10). In the present exemplaryembodiment, the photographing unit 10 is a general camera, andphotographs a subject to generate a digital image. In addition, thephotographing unit 10 is installed on the road side, and performsphotographing from the lateral direction of the vehicle (the directionsubstantially perpendicular to the traveling direction of the vehicle)in the present exemplary embodiment. In order to acquire vehicle imagesat the plurality of timings, the photographing unit 10 may photographthe vehicle in accordance with an externally given trigger, or thevehicle may be continuously photographed at predetermined intervals setin advance. For example, when a laser-type vehicle detection sensor isused to detect a vehicle, the photographing unit 10 may be configured todetermine a timing to start photographing. In addition, it is alsopossible to configure the system such that the speed of the vehicle isdetected by a speed detector of the vehicle, and the photographing unit10 changes a photographing cycle, which is a timing for photographing,according to the speed of the vehicle. Here, the photographing timingmay be any one of a timing to start photographing, a timing to terminatephotographing, and a photographing interval (periodic interval or thelike).

The photographing unit 10 may include an infrared projector in order toclearly photograph the person in the vehicle. In this case, thephotographing unit 10 is capable of photographing light in the infraredrange. Incidentally, the photographing unit 10 may photograph thevehicle so as to transmit only a wavelength in the infrared range usinga band-pass filter in order to reduce the influence of visible light. Inaddition, the photographing unit 10 may include a polarizing filter inorder to suppress reflection of light on a glass surface. Thephotographing unit 10 can mitigate the influence of environmentinformation reflected on the glass surface of the vehicle on detectionby utilizing polarization characteristics of reflected light using thepolarizing filter.

The movement amount calculation unit 11 calculates the movement amountof the vehicle based on the image of the moving vehicle (step S11). Themovement amount calculation unit 11 first detects the specific part ofthe vehicle such as the door handle portion (a door knob or a door outerhandle) from the image acquired by the photographing unit 10, andacquires information such as a coordinate value indicating a position ofthe detected specific part. The specific part of the vehicle may be anyportion, such as a tire, a window frame, a vehicle door, a tail lamp, adoor mirror, and a side mirror, other than the door handle portion aslong as the portion has a characteristic as the specific part of thevehicle. For example, the movement amount calculation unit 11 may detecta license plate, a light, or the like from the image acquired by thephotographing unit 10. However, it is preferable to use a characteristicpart which is close to the human face on the image and easy to detectsuch as the door handle portion. The movement amount calculation unit 11generates positional information of the detected specific part andinformation accompanying the positional information (for example,information indicating whether a tire is a front tire or a rear tire inthe case of the tire) as a specific part detection result.

The movement amount calculation unit 11 associates the specific partdetection results with each other between the images and calculates themovement amount of the vehicle in the image. The movement amountcalculation unit 11 may perform such association for each of twoconsecutive images or collectively for a plurality of images.

When performing the association between two consecutive images, themovement amount calculation unit 11 considers the traveling direction ofthe vehicle. For example, the movement amount calculation unit 11searches whether or not the specific part is detected in an image in thetraveling direction from a position where the specific part has beendetected in the previous image based on the specific part detectionresult. In this manner, the movement amount calculation unit 11 obtainsthe specific part that is associated between the latest image and theprevious image.

An angle of view of the photographing unit 10 (camera) is fixed in thepresent exemplary embodiment. Thus, the movement amount calculation unit11 can predict a direction (trajectory) in which the specific part movesin the image. Accordingly, the movement amount calculation unit 11searches whether or not a detection result of the specific part ispresent in the direction in the next image, and performs theassociation. The moving direction of the specific part at each positionof the image may be manually given. Alternatively, the movement amountcalculation unit 11 may perform the association between the images basedon images obtained by photographing the vehicle subjected to test run atlow speed, and acquire the moving direction of the specific part at eachposition of the image. The movement amount calculation unit 11 can usevarious methods, such as template matching for each partial region and amethod of calculating local characteristic quantities, such as ascale-invariant feature transform (SIFT) characteristic, and associatingthe characteristic quantities with each other, as the method ofassociation between the images.

FIG. 3 is an explanatory diagram illustrating calculation of anestimated movement amount performed by the movement amount calculationunit 11. In the example illustrated in FIG. 3, the movement amountcalculation unit 11 uses the door handle portion as the specific part.FIG. 3 illustrates an image frame t at time t and an image frame t+1 attime t+1. Then, the movement amount calculation unit 11 sets a distanceof the door handle portion between the time t and the time t+1 to amovement amount 1_(t, t+1) of the vehicle.

The depth distance calculation unit 12 calculates the depth distance ofthe face of the passenger of the vehicle based on the movement amount ofthe vehicle calculated by the movement amount calculation unit 11 (stepS12). In order to calculate the depth, the depth distance calculationunit 12 measures a direction of a face of a target person relative tothe camera (photographing unit 10). The depth distance calculation unit12 uses, for example, a pinhole camera model in order to measure thedirection.

FIG. 4 is an explanatory diagram illustrating the pinhole camera model.A distance from an imaging plane to a lens in a case where the cameraphotographs a photographing target A is denoted by f (focal length). Inaddition, when a distance from a center of the imaging plane (a positionfacing a lens center) to A where the photographing target A is projectedis denoted by m, tan θ=f/m. That is, the depth distance calculation unit12 can calculate θ based on m and f.

FIG. 5 is an explanatory diagram illustrating a method of calculating adepth distance from a camera to a target face. In the exampleillustrated in FIG. 5, seats are installed at the front part and therear part inside the vehicle, and a plurality of persons are on board.FIG. 5 illustrates the method of calculating the distance from thecamera to the target face using the principle of triangulation when itis assumed that not the vehicle but the camera moves. As illustrated inFIG. 5, a distance from the camera to the target face at time t isdenoted by d_(t) ^(i), and a direction is denoted by θ_(t) ^(i). Adistance from the camera to the target face at time t+1 is denoted byd_(t+1) ^(j), and a direction is denoted by θ_(t+1) ^(j). Then, when avehicle movement amount from time t to time t+1 is denoted by1_(t, t+1), Formula (1) is established according to the sine theorem.

$\begin{matrix}\left\lbrack {{Formula}\mspace{14mu} 1} \right\rbrack & \; \\{\frac{d_{t + 1}^{j}}{\sin\;\theta_{t}^{i}} = {\frac{d_{t}^{i}}{\sin\;\theta_{t + 1}^{j}} = \frac{l_{t,{t + 1}}}{\sin\left( {\pi - \theta_{t}^{i} - \theta_{t + 1}^{j}} \right)}}} & (1)\end{matrix}$

The depth distance calculation unit 12 can calculate d_(t) ^(i) andd_(t+1) ^(j) by substituting a value calculated by the movement amountcalculation unit 11 for the vehicle movement amount 1_(t, t+1) ofFormula (1) and calculating θ_(t) ^(i) and θ_(t+1) ^(j) using the methodillustrated in FIG. 4. A depth distance D illustrated in FIG. 5 is adistance from the camera to the target face in the directionperpendicular to the traveling direction of the vehicle. The depthdistance calculation unit 12 can calculate the depth distance D asillustrated in Formula (2). The depth distance calculation unit 12outputs this distance D to the passenger number determination unit 13.[Formula 2]D=d _(t+1) ^(j) sin θ_(t+1) ^(j) =d _(t) ^(i) sin θ_(t) ^(i)  (2)

The passenger number determination unit 13 acquires the face of thepassenger detected from the image, and determines the presence orabsence of erroneous detection based on the distance in the depthdirection of a plurality of detected faces of passengers, and determinesthe number of passengers of the vehicle (step S13). The passenger numberdetermination unit 13 first performs a face detection process on theimage acquired by the photographing unit 10. The passenger numberdetermination unit 13 obtains a portion, which is estimated to include aface of a person by the face detection process, as a partial image. Atthis time, there is a case where the passenger number determination unit13 erroneously detects a portion that is not the face of the person asthe face of the person or acquires two partial images from the sameperson.

Thus, when a depth distance between the partial images is equal to orlonger than a predetermined threshold value, the passenger numberdetermination unit 13 determines that the faces included in therespective partial images are different persons. In addition, when thedepth distance between the partial images is shorter than thepredetermined threshold value, the passenger number determination unit13 determines that any of the partial images is erroneously detected.

The passenger number determination unit 13 may determine whether or notthe faces included in the respective partial images are differentpersons, respectively, using not only the depth distance between thepartial images but also the distance in the traveling direction of thevehicle (traveling direction distance). That is, when the depth distancebetween the partial images is equal to or longer than the predeterminedthreshold value (first threshold value), or when the traveling directiondistance between the partial images is equal to or longer than apredetermined threshold value (second threshold value), the passengernumber determination unit 13 determines that the faces included in therespective partial images are different persons. In addition, when thedepth distance between the partial images is shorter than the firstthreshold value and the traveling direction distance between the partialimages is shorter than the second threshold value, the passenger numberdetermination unit 13 determines that any of the partial images isobtained by erroneous detection. The first threshold value used for thedepth distance and the second threshold value used for the travelingdirection distance may be the same value or different values.

The passenger number determination unit 13 determines the number ofpartial images, determined to include the face of the person, as thenumber of passengers except for the partial images determined to beerroneously detected by the above-described process.

FIG. 6 is a schematic block diagram illustrating a configuration exampleof a computer according to the present exemplary embodiment. A computer1000 includes a CPU 1001, a main storage device 1002, an auxiliarystorage device 1003, an interface 1004, a display device 1005, and aninput device 1006.

The passenger counting device 100 according to the present exemplaryembodiment is mounted on the computer 1000. The passenger countingdevice 100 is stored in the auxiliary storage device 1003 in the form ofa program. The CPU 1001 reads out the program from the auxiliary storagedevice 1003 and expands the program into the main storage device 1002 toexecute the above-described processes according to the program.

The auxiliary storage device 1003 is an example of a non-transitorytangible medium. Other examples of the non-transitory tangible mediummay include a magnetic disk, a magneto-optical disk, a CD-ROM, aDVD-ROM, a semiconductor memory, and the like which are connected viathe interface 1004. In addition, when the program is distributed to thecomputer 1000 via a communication line, the computer 1000 may expand theprogram into the main storage device 1002 and execute theabove-described processes in response to the distribution.

In addition, the program may be configured to implement some of theabove-described processes. Further, the program may be a differentialprogram which implements the above-described processes in combinationwith other programs that have been already stored in the auxiliarystorage device 1003. A processor included in the computer 1000 is notlimited to the CPU 1001, and it may be enough to provide a processorcapable of executing a program. In addition, the computer 1000 includesa circuit.

As described above, the passenger counting system according to thepresent exemplary embodiment determines the number of passengers of thevehicle using the depth information on the passenger, and thus, canaccurately count the number of passengers of the vehicle.

Second Exemplary Embodiment

A passenger counting system according to the present exemplaryembodiment will be described with reference to the drawings. Aconfiguration of the passenger counting system according to the presentexemplary embodiment is the same as the passenger counting systemaccording to the first exemplary embodiment except for the configurationof the movement amount calculation unit 31, and detailed descriptionsother than the movement amount calculation unit 31 will be omitted. FIG.7 is a block diagram illustrating the configuration of the passengercounting system according to the present exemplary embodiment. Thepassenger counting system includes a photographing unit 10 and apassenger counting device 300. In addition, the passenger countingdevice 300 includes a movement amount calculation unit 31, a depthdistance calculation unit 12, and a passenger number determination unit13. The movement amount calculation unit 31 includes a movement amountestimation unit 32 and an error calculation unit 33.

The photographing unit 10 photographs a vehicle and acquires an image.In the present exemplary embodiment, the photographing unit 10 is ageneral camera, and photographs a subject to generate a digital image.

The movement amount calculation unit 31 calculates the movement amountof the vehicle based on the image of the vehicle acquired by thephotographing unit 10. Specifically, the movement amount estimation unit32 first estimates the movement amount of the vehicle based on themovement amount of a specific part of the vehicle such as a handleportion of a door.

Then, the error calculation unit 33 measures a distance from a firstposition to a second position of a face of a specific person for eachtiming based on images of the vehicle at a plurality of timings. Here,photographing can be performed at periodic timings, or photographing canbe performed at variable timings in regard to the timing. For example,it is also possible to detect speed of the vehicle and make the timingvariable on the basis of the speed of the vehicle. Then, the errorcalculation unit 33 estimates an error of the movement amount of thevehicle when an objective function becomes an extreme value using asteepest descent method by setting a function including a differencebetween a distance from the first position to the second position at afirst timing and a distance from the first position to the secondposition at a second timing as the objective function. Here, the firstposition and the second position may be characteristic points such as anose, an inner corner of an eye, an outer corner of an eye, and a mouthconstituting the face. Further, the distance from the first position tothe second position may be a distance between arbitrary two points amongthe characteristic points such as the nose, the inner corner of the eye,the outer corner of the eye, and the mouth constituting the face.Further, a relationship between the first timing and the second timingneeds to be the timing at which the specific person is reflected in theimages obtained at both the timings. In addition, the movement amount iscalculated using the plurality of images in the present exemplaryembodiment, but is not particularly limited. Specifically, it is alsopossible to adopt a technique of acquiring the movement amount usingonly one image. For example, it is possible to calculate the movementamount using an afterimage (motion blur) in one image.

The depth distance calculation unit 12 calculates a distance (depthdistance) in a depth direction of the face of the passenger of thevehicle in real space based on the movement amount of the vehiclecalculated by the movement amount calculation unit 31. Specifically, thedepth distance calculation unit 12 calculates the distance in the depthdirection from the photographing unit 10 to the face of the passengerbased on the calculated movement amount of the vehicle and a directiontoward the face of the passenger of the vehicle from a position of thephotographing unit 10 that has photographed the vehicle.

The passenger number determination unit 13 detects the face of thepassenger of the vehicle from the image of the vehicle acquired by thephotographing unit 10, determines presence or absence of erroneousdetection based on a distance in the depth direction between a pluralityof detected faces of passengers, and determines the number of passengersof the vehicle. For example, when the distance in the depth directionbetween the plurality of detected faces of passengers is equal to orlonger than a first threshold value or when a distance in the travelingdirection between the plurality of faces of passengers is equal to orlonger than a second threshold value, the passenger number determinationunit 13 determines that the plurality of faces of passengers are facesof different persons. The passenger number determination unit 13 may usea distance between partial images from which the face is detected as thedistance between the faces.

Next, an operation of the passenger counting system according to thepresent exemplary embodiment will be described. FIG. 8 is a flowchartillustrating the operation of the passenger counting system according tothe present exemplary embodiment.

The photographing unit 10 photographs a moving vehicle at a plurality oftimings and acquires images (step S10).

The movement amount estimation unit 32 estimates the movement amount ofthe vehicle based on the image of the moving vehicle (step S11 a). Themovement amount estimation unit 32 first detects a specific part of thevehicle such as a door handle portion (a door knob or a door outerhandle) from the image acquired by the photographing unit 10, andacquires information such as a coordinate value indicating a position ofthe detected specific part. Then, the movement amount estimation unit 32sets a distance of the door handle portion between time t and time t+1to an estimated movement amount L_(t, t+1) of the vehicle. However, themovement amount on the image differs depending on the depth distance,and thus, there is a possibility that an error may occur in the case ofcalculating a depth distance of a passenger to be described later usingthe estimated movement amount L_(t, t+1) calculated based on themovement amount of the door handle portion. Incidentally, an operationof the movement amount estimation unit 32 according to the presentexemplary embodiment is the same as the operation of the movement amountcalculation unit 11 according to the first exemplary embodiment, andthus, a detailed description thereof will be omitted.

The error calculation unit 33 estimates the error of the movement amountof the vehicle using the steepest descent method (step S11 b).Hereinafter, a method of estimating the error of the movement amountcalculated by the error calculation unit 33 will be described. A vehiclemovement amount 1_(t, t+1) in consideration of the error is expressed bythe following Formula (3). Here, ΔL is a predetermined parameter, and xis an estimation parameter indicating an error of a movement amount of avehicle. In order to calculate the vehicle movement amount, it isnecessary to estimate this error x.

$\begin{matrix}\left\lbrack {{Formula}\mspace{14mu} 3} \right\rbrack & \; \\{l_{t,{t + 1}} = {L_{t,{t + 1}}\left\lbrack {1 + {\Delta\; L\left\{ {\frac{2}{1 + {\exp\left( {- x} \right)}} - 1} \right\}}} \right\rbrack}} & (3)\end{matrix}$

FIG. 9 is an enlarged view of a face of a passenger of a vehicle. Asillustrated in FIG. 9, a distance from an observation point (a positionof the photographing unit 10 in the present exemplary embodiment) to theface of the specific person riding in the vehicle at time t is denotedby d_(t) ^(i), the distance from the first position to the secondposition of the face of the specific person is expressed as d_(t) ^(i)sin α_(t) ^(i). In the example illustrated in FIG. 9, theabove-described distance from the first position to the second positionis a distance from the outer corner of the eye to a distal end of thenose on a profile of the face, but may be another distance as long as itis a distance between predetermined parts of the same person. The errorcalculation unit 33 calculates the above-described distance from thefirst position to the second position of the specific person in each ofan image frame t and an image frame t+1.

A function φ_(t, t+1) ^(i,j), which includes a difference between thedistance d_(t) ^(i) sin α_(t) ^(i) from the first position to the secondposition of the specific person in the image frame t and a distanced_(t+1) ^(j) sin α_(t+1) ^(i) from the first position to the secondposition of the specific person in the image frame t+1, is expressed bythe following Formula (4). Here, i is a variable to specify the facedetected in the image frame t, and j is a variable to specify the facedetected in the image frame t+1.[Formula 4]φ_(t,t+1) ^(i,j) =∥d _(t) ^(i) sin α_(t) ^(i) −d _(t+1) ^(j) sin α_(t+1)^(j)∥²  (4)

In addition, an objective function E configured to estimate the error xis expressed by Formula (5). In Formula (5), σ is a predeterminedparameter. Further, G_(j∈t, t+1) ^(i) is a detection result set on theimage frame t+1 which is a pair candidate with an i-th detection resultof the image frame t.

$\begin{matrix}\left\lbrack {{Formula}\mspace{14mu} 5} \right\rbrack & \; \\{E = {\sum\limits_{i}{\sum\limits_{j = G_{{j \in t},{t + 1}}^{i}}{\exp\left( {- \frac{\varphi_{t,{t + 1}}^{i,j}}{\sigma^{2}}} \right)}}}} & (5)\end{matrix}$

In the image frame t and the image frame t+1, the distance from thefirst position to the second position is the same as long as the personis the same, and the function φ_(t, t+1) ^(i,j) becomes the minimum. Inaddition, E is a Gaussian function, and an extreme value of E is a valuethat minimizes φ_(t, t+1) ^(i,j). Since the movement amount that needsto be calculated is the movement amount of the same person, it ispossible to estimate that a value of x at which E becomes the extremevalue is an actual error. In the present exemplary embodiment, the errorcalculation unit 33 uses the steepest descent method in order tocalculate the extreme value of E.

A method of estimating x using the steepest descent method will bedescribed. In Formula (6), ρ is a parameter to determine a weight of anumerical value to be updated once. Further, s indicates the number ofrepetitions. The error calculation unit 33 estimates the error x whenthe objective function E becomes the extreme value by repeatedlyperforming the calculation of Formula (6). Specifically, the errorcalculation unit 33 sets a predetermined initial value to x(s) at first,and calculates the next x(s+1) using x(s+1) thus calculated as the nextx(s). The error calculation unit 33 repeats the process of calculatingx(s+1), and sets x(s+1) obtained when ∂E/∂x becomes zero or becomes avalue smaller than a predetermined value as a solution of the error x.The movement amount calculation unit 31 can calculate the vehiclemovement amount 1_(t, t+1) in consideration of the error by substitutingthe calculated error x into Formula (3).

$\begin{matrix}\left\lbrack {{Formula}\mspace{14mu} 6} \right\rbrack & \; \\{{{x\left( {s + 1} \right)} = {{x(s)} = {\rho\frac{\partial E}{\partial x}}}}}_{x = {x{(s)}}} & (6)\end{matrix}$

The error calculation unit 33 may estimate the error x using anothermethod, for example, the Newton method, the EM algorithm, or the likeand calculate the vehicle movement amount 1_(t, t+1) in consideration ofthe error.

The depth distance calculation unit 12 calculates the depth distance ofthe face of the passenger of the vehicle based on the movement amount ofthe vehicle calculated by the movement amount calculation unit 31 (stepS12). In order to calculate the depth, the depth distance calculationunit 12 measures a direction of a face of a target person relative tothe camera (photographing unit 10). The depth distance calculation unit12 uses, for example, a pinhole camera model in order to measure thedirection.

The passenger number determination unit 13 acquires the face of thepassenger detected from the image, and determines the presence orabsence of erroneous detection based on the distance in the depthdirection of a plurality of detected faces of passengers, and determinesthe number of passengers of the vehicle (step S13).

Incidentally, a configuration example of the computer of the passengercounting system according to the present exemplary embodiment is thesame as that of the passenger counting system according to the firstexemplary embodiment (see FIG. 6).

As described above, the passenger counting system according to thepresent exemplary embodiment estimates the error of the movement amountof the vehicle using the steepest descent method, and thus, it ispossible to acquire the movement amount in consideration of the errorand the depth information.

Third Exemplary Embodiment

A vehicle movement amount calculation device according to the presentexemplary embodiment will be described with reference to the drawings. Afunction of a vehicle movement amount calculation device 400 accordingto the present exemplary embodiment is the same as that of the movementamount calculation unit 31 according to the second exemplary embodiment,and thus, a detailed description thereof will be omitted. FIG. 10 is ablock diagram illustrating a configuration of the vehicle movementamount calculation device 400 according to the present exemplaryembodiment. The vehicle movement amount calculation device 400 includesa movement amount estimation unit 42 and an error calculation unit 43.

The movement amount estimation unit 42 acquires an image of a vehicleand estimates a movement amount of the vehicle based on the acquiredimage of the vehicle. Specifically, the movement amount estimation unit42 estimates the movement amount of the vehicle based on the movementamount of a specific part of the vehicle such as a handle portion of adoor.

The error calculation unit 43 estimates an error of the movement amountof the vehicle when an objective function becomes an extreme value usinga steepest descent method by setting a function including a differencebetween a distance from a first position to a second position at a firsttiming and a distance from the first position to the second position ata second timing as the objective function.

The vehicle movement amount calculation device 400 corrects the movementamount estimated by the movement amount estimation unit 42 using theerror calculated by the error calculation unit 43, thereby calculatingthe movement amount of the vehicle in consideration of the error.

Next, an operation of the vehicle movement amount calculation deviceaccording to the present exemplary embodiment will be described. FIG. 11is a flowchart illustrating the operation of the vehicle movement amountcalculation device according to the present exemplary embodiment.

The movement amount estimation unit 42 estimates the movement amount ofthe vehicle based on the image of the moving vehicle (step S11 a). Themovement amount estimation unit 42 first detects a specific part of thevehicle such as a door handle portion (a door knob or a door outerhandle) from the acquired image, and acquires information such as acoordinate value indicating a position of the detected specific part.The specific part of the vehicle may be any portion, such as a tire, awindow frame, a vehicle door, a tail lamp, a door mirror, and a sidemirror, other than the door handle portion as long as the portion has acharacteristic as the specific part of the vehicle.

The error calculation unit 43 estimates the error of the movement amountof the vehicle using the steepest descent method (step S11 b). The errorcalculation unit 43 estimates an error of the movement amount of thevehicle when an objective function becomes an extreme value using asteepest descent method by setting a function including a differencebetween a distance from a first position to a second position at a firsttiming and a distance from the first position to the second position ata second timing as the objective function.

Incidentally, a configuration of a computer of the vehicle movementamount calculation device according to the present exemplary embodimentis the same as that of the passenger counting system according to thefirst exemplary embodiment (see FIG. 6).

As described above, the vehicle movement amount calculation deviceaccording to the present exemplary embodiment estimates the error of themovement amount of the vehicle using the steepest descent method, andthus, it is possible to calculate the movement amount in considerationof the error. In addition, it is possible to accurately count the numberof passengers of the vehicle when the movement amount of the vehicle isused to count the number of passengers of the vehicle.

FIG. 12 is a block diagram illustrating a configuration of a main partof the passenger counting system according to the present invention. Thepassenger counting system includes a photographing means 20 forphotographing a vehicle and acquiring an image, and a passenger countingdevice 200. The passenger counting 200 includes: a movement amountcalculation means 21 for calculating a movement amount of a vehiclebased on an image of the vehicle; a depth distance calculation means 22for calculating a distance in a depth direction of a face of a passengerof the vehicle based on the movement amount of the vehicle; and apassenger number determination means 23 for detecting the face of thepassenger of the vehicle from the image and determining the number ofpassengers of the vehicle based on distances in the depth direction of aplurality of detected faces of the passengers.

In addition, the passenger counting system illustrated in the following(1) to (6) is also disclosed in the above-described exemplaryembodiments.

(1) The passenger counting system may be configured such that thepassenger number determination means (for example, the passenger numberdetermination unit 13) determines presence or absence of erroneousdetection based on the distance in the depth direction between theplurality of detected faces of passengers.

(2) The passenger counting system may be configured such that themovement amount calculation means (for example, the movement amountcalculation unit 31) estimates the error of the movement amount of thevehicle using the steepest descent method.

(3) The passenger counting system may be configured such that themovement amount calculation means (for example, the movement amountcalculation unit 31) measures the distance from the first position tothe second position of the face of the specific person for each timingbased on the images of the vehicle at the plurality of timings, andestimates the error of the movement amount of the vehicle when theobjective function becomes the extreme value using the steepest descentmethod by setting the function including the difference between thedistance at the first timing and the distance at the second timing asthe objective function.

(4) The passenger counting system may be configured such that thepassenger number determination means (for example, the passenger numberdetermination unit 13) determines that the plurality of faces ofpassengers are the faces of different persons when the distance in thedepth direction between the plurality of detected faces of passengers isequal to or longer than the first threshold value. According to such apassenger counting system, it is possible to more accurately count thenumber of passengers.

(5) The passenger counting system may be configured such that thepassenger number determination means (for example, the passenger numberdetermination unit 13) determines that the plurality of faces ofpassengers are the faces of different persons when the distance in thedepth direction between the plurality of detected faces of passengers isequal to or longer than the first threshold value or when the distancein the traveling direction between the plurality of faces of passengersis equal to or longer than the second threshold value.

(6) The passenger counting system may be configured such that the depthdistance calculation means (for example, the depth distance calculatingsection 12) calculates the distance in the depth direction from aphotographing means to the face of the passenger based on the calculatedmovement amount of the vehicle and the direction toward the face of thepassenger of the vehicle from the position of the photographing means(for example, the photographing unit 10) that has photographed thevehicle.

As above, the invention of the present application has been describedwith reference to the exemplary embodiments, but the invention of thepresent application is not limited to the above-described exemplaryembodiments. Various modifications that can be understood by the personskilled in the art can be made within a scope of the invention of thepresent application regarding the configuration and the details of theinvention of the present application.

REFERENCE SIGNS LIST

-   10 Photographing unit-   11, 31 Movement amount calculation unit-   12 Depth distance calculation unit-   13 Passenger number determination unit-   20 Photographing means-   21 Movement amount calculation means-   22 Depth distance calculation means-   23 Passenger number determination means-   32, 42 Movement amount estimation unit-   33, 43 Error calculation unit-   100, 200 Passenger number counting device

The invention claimed is:
 1. An erroneous detection determination systemcomprising: at least one memory storing instructions; and at least oneprocessor coupled to the at least one memory and configured to executethe instructions to: acquire a movement amount of a vehicle; calculate,based on the movement amount of the vehicle, distances in a depthdirection of objects detected as faces of passengers of the vehicle froman image of the vehicle; and determine whether the detected objectsinclude an object that has been erroneously detected, based on thedistances in the depth direction of the objects.
 2. The erroneousdetection determination system according to claim 1, wherein the atleast one processor is configured to calculate the distances in thedepth direction of the objects based on the movement amount of thevehicle and two or more images which include objects, the two or moreimages being generated by imaging the vehicle two or more times.
 3. Theerroneous detection determination system according to claim 1, whereinthe at least one processor is configured to determine that the detectedobjects include an object that has been erroneously detected when adistance in the depth direction between two or more detected objects outof the detected objects is shorter than a predetermined threshold value.4. The erroneous detection determination system according to claim 1,wherein the at least one processor is further configured to determine anumber obtained by subtracting the number of an erroneously-detectedobject from the number of the detected objects to be a number ofpassengers of the vehicle.
 5. The erroneous detection determinationsystem according to claim 1, wherein the at least one processor isfurther configured to acquire the movement amount of the vehicle bycalculating the movement amount based on change in a position of aspecific part of the vehicle, the specific part being chosen from: adoor knob; a door outer handle; a tire; a window frame; a door; a taillamp, a door mirror; a side mirror; a license plate; and a light of thevehicle.
 6. The erroneous detection determination system according toclaim 1, wherein the at least one processor is further configured to:estimate a value indicating an error of the acquired movement amountbased on positions of specific parts of a face of a person in thevehicle, the specific parts being selected from: a nose; an inner cornerof an eye; an outer corner of an eye; and a mouth; and correct theacquired movement amount using the estimated value.
 7. An erroneousdetection method comprising: acquiring a movement amount of a vehicle;calculating, based on the movement amount of the vehicle, distances in adepth direction of objects detected as faces of passengers of thevehicle from an image of the vehicle; and determining whether thedetected objects include an object that has been erroneously detected,based on the distances in the depth direction of the objects.
 8. Theerroneous detection method according to claim 7, comprising calculatingthe distances in the depth direction of the objects based on themovement amount of the vehicle and two or more images which includeobjects, the two or more images being generated by imaging the vehicletwo or more times.
 9. The erroneous detection method according to claim7, comprising determining that the detected objects include an objectthat has been erroneously detected when a distance in the depthdirection between two or more detected objects out of the detectedobjects is shorter than a predetermined threshold value.
 10. Theerroneous detection method according to claim 7, further comprisingdetermining a number obtained by subtracting the number of anerroneously-detected object from the number of the detected objects tobe a number of passengers of the vehicle.
 11. The erroneous detectionmethod according to claim 7, further comprising acquiring the movementamount of the vehicle by calculating the movement amount based on changein a position of a specific part of the vehicle, the specific part beingchosen from: a door knob; a door outer handle; a tire; a window frame; adoor; a tail lamp, a door mirror; a side mirror; a license plate; and alight of the vehicle.
 12. The erroneous detection method according toclaim 7, further comprising: estimating a value indicating an error ofthe acquired movement amount based on positions of specific parts of aface of a person in the vehicle, the specific parts being selected from:a nose; an inner corner of an eye; an outer corner of an eye; and amouth; and correcting the acquired movement amount using the estimatedvalue.
 13. A non-transitory computer-readable storage medium storing aprogram that causes a computer to perform: acquiring a movement amountof a vehicle; calculating, based on the movement amount of the vehicle,distances in a depth direction of objects detected as faces ofpassengers of the vehicle from an image of the vehicle; and determiningwhether the detected objects include an object that has been erroneouslydetected, based on the distances in the depth direction of the objects.14. The storage medium according to claim 13, wherein the program causesthe computer to perform: calculating the distances in the depthdirection of the objects based on the movement amount of the vehicle andtwo or more images which include objects, the two or more images beinggenerated by imaging the vehicle two or more times.
 15. The storagemedium according to claim 13, wherein the program causes the computer toperform: determining that the detected objects include an object thathas been erroneously detected when a distance in the depth directionbetween two or more detected objects out of the detected objects isshorter than a predetermined threshold value.
 16. The storage mediumaccording to claim 13, wherein the program further causes the computerto perform: determining a number obtained by subtracting the number ofan erroneously-detected object from the number of the detected objectsto be a number of passengers of the vehicle.
 17. The storage mediumaccording to claim 13, wherein the program further causes the computerto perform: acquiring the movement amount of the vehicle by calculatingthe movement amount based on change in a position of a specific part ofthe vehicle, the specific part being chosen from: a door knob; a doorouter handle; a tire; a window frame; a door; a tail lamp, a doormirror; a side mirror; a license plate; and a light of the vehicle. 18.The storage medium according to claim 13, wherein the program furthercauses the computer to perform: estimating a value indicating an errorof the acquired movement amount based on positions of specific parts ofa face of a person in the vehicle, the specific parts being selectedfrom: a nose; an inner corner of an eye; an outer corner of an eye; anda mouth; and correcting the acquired movement amount using the estimatedvalue.